The present invention relates generally to visualization of three dimensional and four dimensional data and more particularly to client-server architecture for visualization of three dimensional and four dimensional data.
Visualization of multi-dimensional data is generally produced as a rendered volume image. Volume rendering is the process of generating a two dimensional (2D) image from a three dimensional (3D) dataset (e.g., a volume comprised of a stack of 2D images) as it would appear from a specified location with specified rendering parameters (e.g., shading, type of projection, etc.). One method of volume rendering is referred to as ray casting as described in Levoy, “Display of Surfaces from Volume Data”, IEEE Computer Graphics and Applications, Vol. 8, No. 3, May 1988, pp. 29-37.
The primary technical obstacle to implementing a system for interactive volume rendering is quickly, efficiently, and locally processing large amounts of data required to render each frame. The large data volume problem is exacerbated by modern medical imaging equipment, which can themselves acquire large amounts of data. For example, the Siemens Somatom Sensation 64 CT scanner, which can acquire slices of 0.33 mm in thickness, five thousand or more slices may be required for a full body scan. The amount of memory required to store such a large volume of data, as well as the processing performance needed to render it, exceeds the capabilities for typical local processors (e.g., desktop personal computers (PCs), laptops, etc.).
A common approach to circumvent this problem is to utilize a client-server architecture, where a fast server machine with substantial memory and processing power is used to generate a volume rendered image, which is then transmitted over a network to a client machine (e.g., client PC, etc.) where the volume rendered image is displayed. In some instances, a thin client is utilized. That is, the client machine does not require any special processing power and is only used to display images which are entirely computed at the server. Such a thin client architecture is described in U.S. Pat. No. 6,683,933 entitled “Three-dimensional image display device in network” (Saito). One problem with this approach is that it requires a large amount of images to be transmitted from the server to the client. Every time a user interacts with a 3D volume's viewing parameters (e.g., orientation, rotation, translation scaling, color, opacity, shading, etc.) using a pointing device such as a mouse, the client notifies the server about those changes and the server renders a new image and transmits the new image to the client. Using the client-server architecture in such a fashion can be costly and slow due to the large amount of updates.
A common approach to mitigate these problems is to use data compression techniques. The use of compression techniques is limited because, in a clinical setting, the images displayed at the client have to preserve the original diagnostic quality. That is, the integrity of the original image data should not be potentially compromised by the use of lossy compression algorithms. Examples of lossless techniques to transmit compressed images over a network are described in U.S. Pat. No. 4,751,587 entitled “Image recording and reproducing apparatus using differential data compression and expansion techniques” (Asahina) and in U.S. Pat. No. 6,925,117 entitled “Data transmission apparatus, method and program, data reception apparatus and method, and data transmission and reception system, using differential data” (Kamisuwa). These methods rely on differential compression. In differential compression, new data is subtracted from reference data. The resulting difference is compressed and transmitted from the server to the client and the client expands the difference data and subtracts it from the reference data, thereby obtaining the new data. One restriction of differential compression is that the reference data needs to reside both on the client and the server. Even with the use of differential compression techniques to reduce the required transmission bandwidth, it is difficult to achieve interactive performance for multiple clients. In present practice, ten or more rendered frames per second are typically needed to be transmitted over the network for each client. If there are numerous clients connected to a server, the network bandwidth is a significant bottleneck.
Another approach to minimize the amount of data transferred from the server to the client is to use a hybrid combination of local and remote volume rendering, (e.g., a smart client) as described by Engel et al., “Combining local and remote visualization techniques for interactive volume rendering in medical applications”, Proceedings of IEEE Visualization '00. Piscataway, N.J.: Institute of Electrical and Electronics Engineers, 2000; 449-452. In the smart client architecture, the application logic and processing resides on the server and the client has the ability to do local processing as well. The server first transfers a low-resolution, sub-sampled copy of the volume data to the client. When the user interacts with the volume, the smart client performs volume rendering locally using the sub-sampled copy of the volume, without needing to request any work to be performed by the server. Upon completion of user interaction (e.g., the user stops moving the mouse and/or releases the mouse button, etc.), the smart client sends a request to the server to render one image and send it over the network. While the number of images rendered by the server is greatly reduced in such an approach and the interactive performance is improved, the images rendered locally by the smart client during interaction are computed using low resolution sub-sampled data and the resulting images do not preserve the original diagnostic quality.
Accordingly, systems and methods are required that can solve all the aforementioned issues related to thin client, smart client, and differential compression techniques and can provide a solution that has fast performance during interactive volume rendering on the client while visualizing images of full diagnostic quality and minimizing the amount of network bandwidth needed to have a server transmit images to multiple clients.